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Abstract: In a timely essay, the author reminds us that launching an accountability 
initiative without careful thought to how it will affect behavior can do more harm than 
good. 



Essay: 

Accountability is in the air, and the news, these days. In response to various "common- 
sense" proposals to fix problems in education, a friend of mine used to say, "If you thi nk 
there's a simple solution, you don't understand the problem." Recent accountability 
proposals show how true this is. In any accountability regime, it's not sufficient to simply 
select a set of performance measures. It's equally important to consider how the system 
will affect behavior. A well-designed accountability system motivates substantive 
change, not merely gaming the system. And the last thing you want is a system that 
undermines useful diagnostic tools in the name of accountability. 

In January, the National Center for Education Statistics proposed some additions to the 
mass of data that it gathers annually from colleges and universities. Normally an arcane 
subject, to be sure. But buried in the proposal was a provision — clearly motivated by the 
Secretary of Education's Commission on the Future of Higher Education — that could 
seriously hamper current efforts to improve college quality. Russ Whitehurst, director of 
the Department of Education's Institute for Education Sciences (which houses the 
statistics agency), subsequently offered vague assurance that the most damaging of these 
proposals would probably not be implemented. Eef s hope he's right. 

The last twenty years have seen calls for greater accountability by higher education, 
accompanied by growing influence of college rankings by U.S. News and World Report. 
College officials complain that the rankings, which purport to measure college quality. 



improperly emphasize inputs and resourees rather than what happens on eampus. But in 
response to aeeountability demands, they argue that the work of their institutions is too 
eomplex, too varied, and too ephemeral to be redueed to simple output measures. 
Although there is merit to both elaims, the quest to improve eollege quality is far from 
hopeless. 

Several relatively new eollege-quality initiatives show sueh promise that they were 
named by the Seeretary's Commission. Colleges and universities partieipating in these 
projeets have aeeess to sophistieated assessments of effeetive edueational praetiees (from 
the National Survey of Student Engagement, or NSSE, and its eommunity eollege 
eounterpart, CCSSE) and of their students' eritieal thinking, analytie and writing skills 
(from the Collegiate Eearning Assessment, or CEA). NSSE and CEA send partieipating 
institutions eonfidential reports showing how they perform relative to their peers; CCSSE 
posts results on its website. This is valuable information that presidents, deans, 
department ehairs, and faeulty members ean — and do — use to improve the quality of 
eollege edueation. 

But the Commission and the Seeretary want more information that students and parents 
ean use to eompare institutions. The Seeretary often eomplains that she has aeeess to 
more eomparative information when buying a ear than when investing in her ehildren's 
eollege edueation. 

So the statisties ageney proposed adding an "aeeountability" seetion to its annual 
eompilation of eollege and university data. In the first phase, eolleges would be asked 
whieh assessments they partieipate in, whether they post the results online, and the 
eorresponding Web address. So far, so good — many institutions post this information, 
and this would make it easier to find. The misehief begins in the seeond phase, wherein 
institutions report the assessments they participate in and their "score" on each one. 
Knowing which assessments a college uses is a good idea, but reporting scores to the 
government will do far more harm than good. 

Why? Eef s set aside the problem of reducing complex assessments to a single institution- 
wide score. (If you had one score for every auto maker, would that help you choose the 
best station wagon?) The real danger is transforming a diagnostic exercise into grading 
and ranking. It's one thing for college officials to have a confidential report from a 
sophisticated assessment identifying where improvement is needed. It's quite another 
when that information is made public; the emphasis shifts quickly from diagnosis to 
damage control (although CCSSE results are public, community colleges do not compete 
in national and regional markets the way four-year institutions do). And recall that these 
are voluntary assessments that institutions pay to participate in. If your doctor and 
financial planner posted your physical and fiscal health on the Web, would you see them 
more often? Would you see them at all? 

It doesn't take much in the way of critical thinking skills to see where this leads. If the 
Department doesn't produce rankings, others will. In NSSE's case, students' survey 
responses will determine their college's standing, and by extension, the value of their 




degree. So they will act in their own self-interest to make their college look good, 
compromising the fundamental requirement for useful information: candor. More likely, 
though, colleges will simply opt out, as they surely will for performance-based 
assessments like CLA, because participation would risk too severe a public-relations 
penalty. Thus would an ill-conceived push for consumer information drive colleges away 
from the most promising assessment and improvement initiatives in decades. 

Higher education institutions must systematically assess and improve their performance. 
But not all diagnostic information is suitable for accountability and consumer 
information, and a ham-fisted approach like this could sabotage important efforts to 
diagnose and improve colleges and universities. 
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